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MEASURING THE EDUCATIONAL ACHIEVEMENT OF UNDERGRADUATES: 
STATE AND NATIONAL DEVELOPMENTS 



The single greatest concern about the quality of American elementary and secondary 
education in the past few years has been the declining cognitive achievement of school 
children* It is not surprising then, that much of the educational reform movement aimed 
at the schools calls for better measurement of student achievement through standardized 
exams; for establishing minimum competency standards for high school graduation; and 
for comparing student performance across school districts, regions and states. 

Now we are beginning to see similar concerns about student performance at the college 
level. To date,the effects of school reform on colleges and universities have been felt 
primarily in the admissions office. States have encouraged, and in some cases mandated, 
more stringent standards for admission and an emphasis on college preparatory curricula 
in the schools. 

The concern for quality in higher education has earlier origins, however. Throughout the 
1970s and early 1980s there was a growing awareness of the increased amount of 
remediation taking place. This perception of declining achievement in colleges and 
univt. itles was reinforced by disturbing reports of the performance of graduates on 
licensing exams, especially in such fields as teaching and nursing* Grade inflation, 
rampant in the 1970s, also contributed to the loss in public confidence in traditional 
measures of student achievement. 
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By the 1980s, "outcomes measurement," long discussed in higher education circles, took 
on a new urgency and a different emphasis* While many institutions had developed 
measures of "client satisfaction" — for example, student ratings of faculty, alumni 
surveys of the lf v3iue" of college — lew had developed assessment instruments that 
measured academic achievement that could be compared either over time or to national 
norms. At the same time, there were moves to include outcomes measurement as a 
legitimate requirement for accreditation* In 1983, the Southern Association of Colleges 
and Schools (SACS) proposed to its members that institutions be required to evaluate 
student learning systematically and measure appropriate outcomes of the education 
process. This represented a significant change from the "process^ oriented" criteria that 
dominate institutional accrediting procedures. While the proposal was not adopted, the 
accrediting community is continuing to debate the appropriate use of outcomes 
measurement in the accreditation process. 

In 1983, Ernest Boyer, after completing his study of American high schools for the 
Carnegie Foundation, suggested that if colleges had the equivalent of the SAT exam for 
their seniors, the results would reflect the same declines as seen in the schools. Then in 
the fail of 1984, the National Institute of Education (NIE) Study Group on the Conditions 
of Excellence in Higher Education called upon faculty and academic deans "to implement 
a systematic prograA to assess the knowledge, capacities, and skills" of college 
students.* 



National Institute of Education Involvement in Learning Realizing the Potential of 
American Higher Education (Washington, D.C: U.S. Department of Education, 1984). 
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Some institutions are already tackling the problem. They have begun to monitor grade 
inflation and tighten retention standards. Others are making curriculum changes — for 
example, greater emphasis on writing. ("Writing Across the Curriculum" programs have 
gained in popularity.) Other institutions are monitoring their students' progress through 
"value-added" approaches that provide pre- and post-/tests assessments. There is also 
renewed interest in comprehensive senior exan\s as 4 requirement for the baccalaureate. 

The use of standardized exams for assessment purposes remains limited, however. The 
following are examples of relatively new uses of tests for three distinct purposes: i) 
program evaluation; 2) student placement; and 3) student progression. In most jpses 
these initiatives have originated at the state and/ or system level. 

Program Evaluation 

Student achievement as a criterion for judging programs is finding its way into statewide 
program evaluation. In Tennessee it is an explicit part of the statewide instructional 
evaluation program, initiated by the Tennessee Higher Education Commission, which 
, offers institutions financial incentives for program evaluation. For example, Tennessee 
Tech earned an additional $767,000 in state support in i' $4, in part because it used 
the ACT-COMP exam to evaluate general education skills of seniors (and then compared 
that achievement to predicted scores to measure value added). The institution also used 
an undergraduate test in business, developed by the Educational Testing Service (ETS), 
the National Teachers Exam, and alumni surveys to assess the relationship between 
academic preparation and later employment. Other state systems (e.g. Georgia) are 
using the pass rates on professional licensing exams as indicators oj^ quality, most 



commonly in the teaching and nursing fields,' where high failure rates are used to justify 
program discontinuance. 

t 

Student Placement ' 

Standardized test scores have ior.g been used by institutions for counseling and course 
placement. Now, initiatives coming from the state level would refine and strengthen this 
effort through new instruments and mandated participation from public institutions*. The 
New Jersey Basic Skills Test is a good example. In 1977, following a year-long study by a 
blue ribbon task force, the New .lersey Department of Higher Education established the 
"Basic Skills Assessment Program" (BSAP) which mandated a basic skills test to evaluate 
a'.i public college students — both full-time and part-time. The exam, developed in 
cooperation with the College Board and under contract with ETS, was first administered 
in 4978. In its sixth form now, the exan consists of an essay and four multiple-choice 
sections: reading comprehension, sentence sense, computation, and elementary algebra. 
Ail entering freshman and transfers with less than 12 credits must take the test, which is 
used for course placement and counseling, not admissions. The test is administered in all 
public colleges and universities, and eleven of the state's private institutions voluntarily 
participate in the program. The Basic Skilis Council, a group of 12 faculty members and 
administrators representing the various sectors of the state higher education system, 
provides policy direction for the testing program. Council members are appointed fo> 
two-year terms by the department of higher education. The costs of th» program include 
the contract with ETS for generating the test items, printing and scoring (approximately 
$500,000 annually for 50,000 to 60,000 test-takers) and administrative support from the 
.apartment of higher education (a staff of four responsible for report- writing and policy- 
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setting)* ^TS ^^-the College Board have developed similar placement testing programs 
for Florida public institutions and the California Community College System, 

Ohio has taken a different approach to placement testing. With financial support from 
the legislature, the Ohio Board of Regents is administering a statewide testing program 
to high school juniors in 600 of the state's 900 high schools. It is a voluntary program 
linked closely to the state's public colleges and universities. High school juniors are 
examine S^n writing, science "readiness 11 and mathematics skills. In thwp case of 
mathematics, students receive feedback from the Ohio college of their choice as to their 
likely placement in math sequences and their eligibility to pursue particular majors. This 
allows students to take corrective action in their senior year. At the same time, college 
faculty members have agreed to serve as resources to the school in improving 
curriculum. The effect on student readiness for college-level work has been significant. 
Ohio State University, for example, has seen the 5 percentage of freshman requiring math 
remediation drop from 50% to about 35% since the implementation of the program. 
State support for the high school testing program is approximately $500,000 annually. 

Student Progression 

Standardized exams to establish minimum standards for student progression in higher 
education are a limited but growing response to concerns about quality. Some systems 
are usin^ these exams to screen students seeking admission to teacher education 
programs,- and as a requirement for ail undergraduate students seeking admission to upper 
divisions. 



Mississippi exemplifies states now requiring a test of general education skills (the ACT- 
« tO MP EXAM) for students petitioning for admission into teacher education programs. 
Usually students must take and pass the test at the end of the sophomore year to qualify 
for profess : onai courses. The state's board of trustees adopted the exam in large part to 
assure the public that only competent students will be allowed to study to be teachers. 
Costs of the program, except for on-site administration, are borne by the student ($25 
per student in 1983). Students may re-take the exam as often as they wish. Failure ra :es 
are 45% to 50% per administration. 

Two states have gone as far as requiring all students to pass a minimum competency 
exam for advancement to upper-division programs. These are the so-called "rising 
junior" exams. The Georgia Board of Regents requires all students in the system to pass 
its "Regents Exam" (which includes a written essay) before graduation. Students, who 
first take the exam at the sophomore level, may re-take it until they pass it. In Florida, 
a sophomore exam entitled the "College Level Academic Skills Test" (CLAST) was 
developed by the department of education as a requirement for advancement to upper- 
division programs. Pass rates range from 75% to 80% on the first try. The Florida 
Postsecondary Education Planning Commission reports that the basic bank of CLA5T 
items and essay topics cjst the state about $500,00>0 to develop, not including faculty 
time for item review. Florida estimates annual administrative costs for production, 
distribution, and scoring are $500,000, not including costs of managing the program 



^Many states are now requiring testing as part of the certification requirements for 
prospective readers. See "Testing for Teacher Competency" Issue gram No. 7 (Denver, 
Colorado: Education Commission of the States, 1983). • 
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through the department ,of education, or costs to the institutions for administering the 
exam. A few other states, such as New 3erse,y and Arkansas, have considered such 
exams. Even without a state mandate, some public universities have initiated their own 
institution-wide or system-wide assessment exams, for exampie, the University of 
Minnesota and the California State College System.-* 

> « 

L egaHssues 

While there is yet no case law relating to minimum competency exams at the 
postsecondary level, states should consider, at least, the dictates established for similar 
exams at the secondary level.** First, case decisions at the secondary level establish that 
students have a constitutional right to adequate notice of a testing program. In Debra P. 
v. Turlington (Florida), less than two-years 1 notice was viewed as violating due process. 
Second, graduation exams must reflect the material taught; "Like employment tests, 
competency tests must be validated for tbsir purposes to be fair."^ Third, and an 
important corollary to content validity, is the requirement that tests not be racially or 
culturally biased. 



For further reading on this subject see John Bepnett et al- "Academic Progression 
Tests for Under graduates: Recent Developments," Educational Record, (Winter 1984): 
M-**6i Measuring Educational Progress in the South: Student Achievement (Atlanta: 
Southern Regional Education Board, 1984). 

^See Christiane H. 'Citron, "Legal Rules for Student Competency Testing," ECS 
Issue gram No. 36 (Denver, Colorado: Education Commission of the States, March 1, 
T983). 

^Ibid., p; 3. 



Legislative -Initiatives ,* 
Few state legislatures have been more aggressive in setting academic and curriculum 
standards than Florida/ Among the rriore controversial initiatives has been the "Gordon- 
Rule/ 1 named after a Florida legislator, which directs institutions to require students to 



complete 12 semester hours of English course work that includes written work of at least 
6,000 word?* It also requires six semester hours of mathematics coursework at the 
college level. Subsequent changes in the rule allow institutions to develop alternative 
plans for assuring that their students have appropriate communication and computation 
skills* Institutions have objected to these and other intrusions into curriculum, but they ' 
have at the same time appreciated the increased financial support for "quality* 
initiatives provided by the legislature* 

- & 

In a 198* supplement to its master plan, the Florida Posts econdary Education Planning 
Commission pinpointed what many believe is the heart of the quality problem in higher 
education — namely, that states need to establish a "clearer sense of educational 
sequence and progression:" 

To have meaning, this progression must be outlined by definitive standards of entry 
and exit at each major transition point in the continuum: higfr school graduation, 
college entry, qualification for upper-level study, admission into graduate and 
professional education. To accompany this process, there is a need for higher 
expectations with regard to minimum student performance* The high school 
diploma in many cases now means no more than the presence of eighth-grade skills 
for some students; college-level academic work has come to imply an^ course taken 
by students in college, not courses which develop and demand learning and thinking 

S 
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skills above the eleventh or twelfth grade level. Since students of ten receive 
Associate of Arts (A .A.) degree credit for work that is below college level, the 
degree is diluted and does not Indicate the attainment of post high school skills 
which are becoming so* critical to the economic and social success of ail people.^ 

The commission went on to recommend that clearly recognizable "bridge" programs be 
developed for students who "are ftot ready to begin a higher level of education even 
though they have met the minimum requirements of the preceding level." 

In 1984 the Fiorina legislature acted on these recommendations by requiring, as a 
condition of entry^ that students be tested for basic computation and communication 

I 

skills, and that th<^se students requiring remediation be enrolled. in "college prep" 

programs in community colleges. (Withojne exception, four-year institutions must 

i ' I , 

contract with community colleges for this college-preparatory instruction. College 

credit is not awarded for this -instruction.) 

The Tennessee legislature's approach has been to put pressure on institutions to quantify 

their program objectives. In 1984 the legislature enacted a bill that established higher 

education improvement goals and /required the Tennessee Higher Education Commission 

/ \ 
to report (within five years) the quantifiable progress on each item. Among the goals 



*The Master Plan for Florida Postsccondary Education: Supplement Number 2 
(Tallahassee, Florida: Postsecondary Education Planning Commission, Department of 
Education, March 17, 1984). 
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listed are 

1) &n Hicrease in the percentage of students who enter four- year university-degree 
programs and who subsequently earn degrees; 

2) an improvement in the average NTE scores of students In teacher-education 
programs; 

3) an improvement in standardized examination scores of graduating seniors at 
pufjfic universities; 

<f) an increase in the number of students from j>ubiic universities who pass ail parts 
of professional licensing exams in such fields as engineering, medicine, law, and 
nursing; and 

♦ • 

5) an improvement in the job placement rate in vocational fields. 



There are fifteen specific goals in all, and options to show improvement on other 

* ■ -> 

unspecified, goaisr , ^ 



National Developments j I 

i 

When the NIE Study Group released its report in October 198^, it noted several "warning 

t t 4 ■ - 

signals" that should alert higher education to present and potential problems. One of 

1/ 

these was that student performance on 11 of 15 major subject-area tests of the Graduate 

* * ! 

Record Exam (G^E) declined between 196£ and f9S2. "The sharpest declines," they 
noted, "occurred in subjects requiring high verbal skills*" 7 While this finding was 



7 ; 
Involvement in Learning, p. 9. 
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criticized by those who objected to using GREs as exit measures, Cliff Adeiman, author 
of a follow-up study lo the NIE report, disagrees. His study, wh'.ch reviewed the 
r ;rf ormance of students on 23 different standardized exams (such as the Graduate 
Medical Admissions Test and the Law School Admissions Test) from 1964 to 1982, fcund 
declines in 15 of the exams, increases in four and four unchanged. He concluded that 
these changes were not attributable to differences in age, gender or race of the test- 
takers, but to the enrollment mix of majors. He infers that increasing specialization of 
curriculum and major selection have dragged down the test scores. 

The increased emphasis on testing has been a boon to the testing industry. Numerous 
instruments are available to colleges and universities and refinements are being 
developed *!:-.! marketed aggressively.^ State boards may be especially interested in the 
following activities aimed at undergraduates, being undertaken at the Educational 
Testing Service (ETS), The American College Test( ACT), and the National Center for 
Education Statistics (NCES). 

1. The Undergraduate Assessment Program developed by ETS several years ago is now 
receiving renewed attention as an instrument in comprehensive senior exam programs. 
The exams correspond to Graduate Record Exams, but are intended for juniors and 



^Clifford Adeiman, "The Standardized Test Scores of College Graduates 1964-1982." See 
NIE Study Group on the Condition of Excellence in American Higher Education 
(Washington, D. C: National Institute of Education, December 1984). 

o 

'For a complete list of exams developed for college age students see James V. Mitchell, 
ed.. Tests in Print III (Lincoln, Nebraska: Lincoln Nebraska Press, 1983). 
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seniors. They are major-specific (eg. art history, biology, business, chemistry); they do 
not measure general education, however, nor can they be made core-specific (eg, there is 
a history exam but no "European history" exam). 

2. Diagnostic Testing for Placfement Purposes. ETS is currently field- testing i 
computerized adaptive test that allows accurate measurement with far fewer questions 
than conventional battery tests. Essentially tests are tailored to the test-taker, yet 
allow for comparison. Test items are similar to those developed for the New Jersey 
Basic Skills Test. 

3. Value-Added Testing. As noted earlier, some institutions are now using the ACT- 
CO MP exam <*nd the ACT high school exam to measure gains in general education 
knowledge and skills among under pyduates. (The subparts of ACT-COMP are entitled 
Functioning in Social Institutions, Using Science, Using the Arts, Clarifying Values, and 
Solving Problems.) The exam requires oral pre sen tat ions. 

4. National Longitudinal Studies. Since 1972, the National Center for Education 
Statistics (NCES) has been following the progress and examining the outcomes of a high 
school cohort in its ongoing study, Hi flh School and Beyond. Plans are now uglier way for 
a new study of the class oi 198S, which for the first time will include a posts econdary 
component (first-time freshmen). Such a longitudinal study will permit, for example, an 
analysis of the nature, extent and reasons for college persistence and withdrawal. It will 
provide a description of the characteristics of students in different fields that includes 
an analysis of their coursework and grades. In two follow-ups, the study also will 

12 



examine labor- force participation and financial success and thus allow analysis of the 
relationship of these outcomes to student characteristics and performance in college. 
Whiie NCES plans to support a national sampling of institutions, it is offering states the 
opportunity to augment the NCES sample and to facilitate accurate state- by-state 
analysis. Costs to the state would depend on the number of institutions needed to lower 
the standard error. ^ 

5. NAEP Developments. The National Assessment of Educational Progress, administered 
by ETS with support from NIE, will begin in the spring of 1985 to test a national sample 
of 21-25 year olds, to assess the degree of "adult literacy." The objective is to move 
away from simplistic characterizations of individuals as "literate" or "nonliterate" an<l 
assess more broadly the ability of adults to read, listen and communicate. NAEP staff 
members also expect to be able to determine the effects of college attendance on 
literacy. 



Conclusion 

National concerns about the quality of undergraduate education and the increased 
activism of state legislatures on education issues clearly are affecting postsecondary 
education. As with the elementary/ secondary schools, much of the focus at the college 
level is on the cognitive outcomes of student learning. In the words of a recent report by 

i0 NCES is charging $>000 per institution for the base survey and two follow-ups. Thus, 
if a state needed to add fifteen institutions to the sample the cost would be $135,000 
payable over six years. For more information, contact David Sweet at NCES. 
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the Southern Regional Education Board: "Today, there is interest in a new form of 
accountability for higher education — accountability on the basis of the demonstrated 
achievement of students, not just on financial criteria; and quality judgments on the basis 
of student academic success, not just on the basis of selectivity."** 

Increasingly, we can expect states and institut/ans to use student performance as a 
legitimate criterion for program evaluation and for legislatures to put pressure on state 
coordinating and governing boards to discontinue programs where students are failing 
licensing exams and performing poorly on other standardized measures* 

Standardized exams are most applicable and least controversial when used for placement 
and counseling. In fact, their use is viewed by many as a necessary ingredient in open 
admissions systems. Ohio's efforts to deal with students 1 deficiencies while they are still 
in high school may be especially productive. It is also encouraging to see examinations 
move beyond the simple and inadequate multiple-choice variety to get at more difficult 
testing problems such as writing skills. 

To date, using state or system-wide exams as mechanisms for determining the individual 
student's eligibility for advancement has not been a widespread practice. Nor have many 
state legislatures tried to prescribe curriculum. (The move by systems of institutions to 
establish a "core" curriculum is gaining momentum, however.) This is, in large part, 
because of faculty and institutional opposition to external intervention into academic and 



M easuring Educational Progress , p.42. 
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curriculum standards. 

State-level intervention Jnto areas traditionally reserved for faculty and mstitutions is 
the result of several factors: the inability of institutions to maintain standards and 
provide expanded access simultaneously; the loss of confidence in the grading practices 
of faculty; the failure of high schools to teach basic computation and communication 
skills; and the desire to extend successful "school-type" reforms to colleges and 
universities. 

For the most part the changes discussed in this paper have been constructive. "Rising 
Junior" exams can be regarded as a new method of establishing admissions standards — 
now at the end of the fourteenth year rather than the twelfth. Such a policy fits well 
with the goal of extending access to at least two years of college to all high school 
graduates, independent of ability. 

Exams, whether for the purposes of program evaluation, student placement, or student 
progression have also drawn public attention to an important postsecondary issue — the 
need for substantially improved remedial programs. If states are to maintain access , and 
at the same time avoid further degrading the meaning of college degrees, remediation 
problems must be faced directly. In some states this attention to the remediation 
problem has led to increased financial support from the legislature. The competition 
inherent in exams also can motivate institutions, departments, and students to perform 
better. 
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"Minimum competency" exams may have deleterious effects, however. Courses may be 

altered to "teach to the test" and result in curriculum less stimulating to more-capable 

students. In 1980, during the debate over "truth in testing," Rexford Brown noted that 

many experts believe that the "formal qualities of multiple-choice tests convey messages 

12 

that undercut reading skills, writing ability, and accurate perception of the world." 
Tests can also, if used only as screening devices, significantly reduce access and 
opportunity for large segments of the population. Failure rates on a variety of 
standardized exams have been consistently higher for minority groups. 

Finally, a standardized examination implies a standardized curriculum — a goal that is 
neither feasible nor desirable in many cases. This is why the testing of minimum 
competency is far less problematic than standardized graduation exams. 

States would be wise to consider a variety of options for encouraging quality 
improvements that institutional leaders and faculty members support. After all, 
governors, legislators, rnd other state officials must depend on those in the classroom for 
implementation. There are many at the institutional level seeking reform who will be 
encouraged by external support. Reform should begin with tightened retention standards 
and grading practices, followed by examination for placement purposes and standards for 
progression. Finally, states may want to explore ways to encourage institutionally 
developed assessment programs that are program-sp- ~if ic and that include a variety of 
instruments. 

12 Rexford Brown, "Searching For the Truth About Truth in Testing Legislation "' R eport 
No. 132 (Denver, Colorado: Education Commission of the States, January, 1980). 
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